fix(minibf): fix ordering and 404 edge cases in account addresses - #1221
Draft
slowbackspace wants to merge 6 commits into
Draft
fix(minibf): fix ordering and 404 edge cases in account addresses#1221slowbackspace wants to merge 6 commits into
slowbackspace wants to merge 6 commits into
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
slowbackspace
force-pushed
the
fix/minibf-account-addresses
branch
from
August 18, 2026 10:29
621fe85 to
d30cf75
Compare
Contributor
|
|
vladimirvolek
force-pushed
the
fix/minibf-account-addresses
branch
from
August 19, 2026 22:26
d30cf75 to
ea597ea
Compare
The /accounts/{stake_address}/addresses endpoint diverged from
Blockfrost in two cases:
- order=desc sorted addresses by their latest on-chain appearance.
Blockfrost returns the exact reverse of the asc list, which orders
addresses by first appearance. Reused addresses came out in the
wrong position.
- Accounts that only appear inside pool registrations (reward account
or pool owner) returned 404. Blockfrost knows these credentials and
returns an empty list.
Fixes #1140
The /accounts/{stake_address}/addresses endpoint scans every archive
block that touches the account. With correct first-appearance ordering,
a desc request must scan the account's full history: 305 seconds
measured on mainnet for an exchange account with 400k+ addresses.
The stake address log stores each (stake credential, address) pair
once, at its first on-chain appearance, ordered by slot, transaction
order, and output order. Both orders become one page read.
The log is a projection of the block history, so it lives in the
archive store beside the archive tags and the exact lookups, and it is
written through the same ArchiveWriter::apply_index call in the same
batch as the blocks.
- core: `ArchiveIndexDelta::stake_addresses` carries the candidates;
`ArchiveStore::addresses_by_stake_log` reads a page and
`mark_stake_log_ready` declares the log complete.
- fjall: new `archive-stake-log` keyspace with pair entries (the write
probe and undo key), ordered entries (the page read) and the ready
marker. The writer keeps a batch-local seen-set because a batch cannot
read its own pending inserts. The keyspace is not swept by
prune_history: its entries are first appearances.
- memory: same semantics, so ToyDomain tests exercise the log.
- noop answers None.
- The apply path emits appearances from `index_block`. The undo path
rebuilds the same deltas, so a rollback removes exactly what apply
inserted, and only when the undone block was the pair's first
appearance.
- Genesis bootstrap marks the log ready, before the state cursor, so a
crash in between re-runs genesis. Stores restored from a stele or
synced before the log answer None and the endpoint falls back to the
archive scan until a resync.
slowbackspace
force-pushed
the
fix/minibf-account-addresses
branch
from
September 11, 2026 09:03
568d3ba to
977e0cd
Compare
Blockfrost declares only count, page and order for
/accounts/{stake_address}/addresses. Neither ryo nor mimicry binds
from/to for it, so a windowed request answers the full list there.
MiniBF honored them through the shared pagination struct and narrowed
the scan, a quiet divergence.
Scan the whole history like the sibling handlers that take no window,
and drop the guard that kept windowed requests off the stake address
log. A test checks that a window starting at the newest first
appearance changes nothing on either the log path or the scan fallback.
The fallback scan for stores without the stake address log walked the account's whole history ascending and reversed it, because a descending walk meets a reused address at its latest appearance first. Measured on a preview store: 20s for an account with 185k tagged blocks whose asc page answers in 70ms. Walk newest-first instead and ask the archive address tag where each address really belongs: its earliest tagged block is its first production, since an address is spent only after it is produced. Each address is looked up once, when the scan first meets it, and emitted when the scan reaches that block. A page of a many-address account now fills from recent blocks and stops; the two preview accounts above drop from 15-21s to under 0.12s with byte-identical bodies. Accounts with a handful of reused addresses still read their whole history in either order, as asc does on main; the stake address log is what answers those in one page read.
The endpoint kept an archive-scan fallback for stores without the stake address log, gated by a ready marker that genesis wrote. The fallback is what the log exists to replace: an ascending scan that reverses for desc reads an account's whole history, and even the newest-first variant does so for accounts with a handful of reused addresses (226s for the heaviest preview account). Either the index is there or the endpoint is not worth serving from a scan. Drop the fallback and the marker: the handler is one log read in either order, `addresses_by_stake_log` answers a page rather than an `Option`, and `mark_stake_log_ready` goes with its seven implementations and the genesis hook. The log is populated by every path that applies blocks from genesis: relay sync, Mithril bootstrap, WAL catch-up, `doctor rebuild-state` and the snapshot backfill. A stele restore does not write it, because the `indexes` stele layer carries tag and exact records only; that is a known gap to close in the stele profile.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1140.
Summary
Fixes two
blockfrost-testsfailures inGET /accounts/{stake_address}/addresses: wrongdescordering and a 404 for pool-only accounts. Also adds a stake address log, hosted in the archive store, that serves the endpoint in O(page) time.Rebased on
mainafter #1301, #1302, #1303 and #1304 retired the standalone index store and redb backends. The log now lives where the other block projections live: the archive store.Bug 1:
order=descreturned the wrong orderdescand kept the first hit per address. This ordered reused addresses by their latest appearance; Blockfrost orders by first appearance (lowesttx_out.idper address, see ryo'saccounts_stake_address_addresses.sql).descis a reverse range read of the same entries, so it is the exact reverse ofascby construction.stake1u9uz4j...reuses its oldest address. All four mainnet fixture cases match a fully synced mainnet snapshot (thedesccase failed before). On the synthetic chain,descequals the reverse ofascand adescpage is a window into the reversed list.Bug 2: pool-only accounts returned 404
stake_test1uzkdwx64...appears on chain only as reward account and owner of three pools. Dolos creates noAccountStatefor it, so the existence guard returned 404; Blockfrost returns200 []because db-sync registers pool reward accounts and owners.PoolStatefor a pool that names the credential as reward account or owner. The scan runs only on the would-be-404 path, so the hot path pays nothing.from/toare ignoredBlockfrost declares only
count,pageandorderfor this endpoint, and neither ryo nor mimicry bindsfrom/toin its query. MiniBF honored them through the shared pagination struct and narrowed the scan. The handler now ignores them and always covers the whole history, like the sibling handlers that take no window.accounts_by_stake_addresses_ignores_from_and_topins this on both the log path and the scan fallback.Stake address log
No scan can serve this endpoint. A correct scan must find every address of the account before it can order them, and it cannot tell an account with four addresses and 821k blocks from one with a fifth address in its last block. Measured on a copy of a preview store (bootstrapped 2026-09-10, five heaviest accounts by stake-tagged block count,
count=100, two runs each, seconds):ascscandescscan (asc + reverse)stake_test1uz2a3lk22…stake_test1ur96gg3s3…stake_test1upv7n2x0l…stake_test1urk948umh…stake_test1uzd7tg4e6…The log column comes from a preview store bootstrapped from scratch with this branch's binary (
dolos bootstrap mithril, 2026-09-11). Every one of the 25 requests answered in 0.4–2.0 ms, and every response body is identical to the scan's on the older store.A smarter scan does not change the picture. A newest-first walk that places each address at its earliest archive
addresstag brings the two many-address accounts to under 0.12 s fordesc, but the three few-address accounts stay where they are, because only reading every block proves there is no further address. A fully synced mainnet snapshot showed 305 seconds for an exchange account with 400k+ addresses under the old scan, which is also a DoS shape on public nodes. So there is no scan fallback: the endpoint reads the log, and only the log.The log stores each
(stake credential, address)pair once, at its first on-chain appearance, ordered by(slot, tx order, output order). Bothascanddescare a single page read (descis a reverse range read).ArchiveIndexDelta::stake_addressesStakeAddressAppearance { order, stake, address }per produced output with a stake credentialArchiveWriter::apply_index/undo_indexArchiveStore::addresses_by_stake_log(stake, offset, limit, reverse)archive-stake-logkeyspace with two entry shapes: a pair entry per(stake, address)(write-path probe and undo key) and an ordered entry (page read). The writer keeps a batch-local seen-set because a batch cannot read its own pending inserts. Not swept byprune_history: entries are first appearances, so removing one below the cutoff would drop an address the account still uses.ToyDomainendpoint tests exercise the log. noop answers an empty page.compute_undorebuilds the archive deltas with the sameindex_blockthe apply path uses, soundo_indexremoves exactly whatapply_indexinserted. A pair is removed only when the undone block is its stored first appearance.doctor rebuild-state, snapshot backfill. Stores that predate the log resync; that is the migration, as for every earlier index dimension.indexesstele layer carries tag and exact records only (discriminants 0 and 1). A node bootstrapped from a stele answers this endpoint from an incomplete log until it resyncs. Closing it means a third record kind in the layer, aPROFILE_VERSIONbump so old stelae are refused, and republished stelae; that is a follow-up on the stele profile, which is itself still landing (its bootstrap docs are docs: a bootstrap page fordolos bootstrap stelae#1217).Testing
accounts_by_stake_addresses_order_descassertsdescequals the reverse ofascand that adescpage is a window into the reversed list.accounts_by_stake_addresses_pool_only_account_returns_empty_listqueries the synthetic pool owner credential with noAccountState; the old implementation fails with 404.accounts_by_stake_addresses_ignores_from_and_topins that a window starting at the newest first appearance changes nothing.stake_log_round_trips_and_pagesintests/archive_index_roundtrip.rsruns against fjall and memory: first-appearance dedup inside one writer and across writers, ordered paging from both ends, offset windows, in-block ordering, stake isolation, and undo of the first appearance only.🤖 Generated with Claude Code